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loadjoai.c simulates an algorithm Cor balancing the load cr. data pad 
by changing the assignments cf devices to paths. 



ns 



Direct questions regarding this program to: 
Hie ha re Defouv 

e_ *ai 1 : Richard_D<szou;v3 3 :cr:e>, . corr. 
telephone: 2C3-6~3-5517 



o?ssa?ioh of program 

1. Read parameters frcm -he command lina. 

•This is dona by mair.() . Tha caramaters specified in tha command line 
drc the following global variables: 

(a) Configuration parameters 
num_oaths » number or car a paths 
num_dev - numb a r cf devices (di3fcsj 

(b) Algorithm parameters (defined with declarations) 
Tl 

T2 

3icve_Ii:tit 

(c) Simulation parameters 

num_triais - number of tirces to execute the balancing algorithm 
loac*. oxocution or the algorithm ia called a trial cr an experiment) 
3aed '» aeed for random number generator 
(c) Mixed (ccaJliguration/siniulatioiii parameters (no be explained shortly) 
dc«v_per_path 
genuine ce 

2 . ?or each trial: 

(a) Craata the initial condition (the state that the balancing algorithm 
is to act cr.). This is done by the function, gor-^valuas ; ) in two staps: 
(i) Decide how many devices are allocated co each path in the initial 
staua. This is dons by first assigning dev_per_pach devices to 
each path and then distributing the remaining devices randomly 

over 

the path?. Thus, the parameter dev_perjath can ce used to control 
hew evenly the devices are distributed over the oaths. If you want 
tha devices distributed as evenlv as tcssible, set devjer^cath = 

fii) Assign a value tc each dsvics . This vaiuQ is the contribution of 

the 

device to the load on the path that is allocated to tha device; 

that 

is, it is Ttroportionai to the amount cf data transferred by the 

device 

during the most recant time segment . The device values are taken 

freer. 

a random distribution; the distribution is determined by the 

oaramacsr 

gsn_mcde (gen^cde = 2 selects an exponential distribution, which 

is 

probably the most realistic of the choices offered) . 

ib) Apply the balancing algorithm. This is done by the function balance ( ) . 
Pseudocode for the algorithm is provided in lcad_bal .dec . 

ic) Update the statistics recorded for the trials performed so far. 



3 Cue put the statistics summarizing tha rosults on all tha trials . Tha nose 
important 

statistic shows hew affQcti|/-5 the path balancing is in reducing the load 
on the r.cst 

heavily used path. 
•/ 

* include <3Zdio . r.> 
t include <math.h> 

^define KAXpaths 100 

*defins MAXdav 2 ZQ /* maximum number of devices on any path * ■' 

/■ The following constants are us ad by the random -number generator ranc._unic ( } 
*/ 

♦define HA 153 07 

Sdefir.e IM 2147433647 

#de£ir.a AM '.i.O/IMi 

^define £Q 12777.3 

tfdefin-a 13 233 5 

frdofino MTA3 3 2 

frdefine MCIY ( I* {IX-l) /NTAfi J 

*deJi.ne £?s l.le-14 

^define RXKX ( 1. G--5PS1 

Mefina laxfA., 3) J (A) > (3) ? (A! ; (3)) 
^define min(A, 3) ((A) < (3) ? (Aj ■ (5)) 

/* ***<****' aSGINNICTG C? EXTERNAL VAP.rA31.2S **************** * * v 

int num_oaths; /- number of cath< */ 

tnc nura_dev ; ./* total numbar of csv^cqs */ 

int dev_per_pach ; /* minimum number of devices per path at beginning of 
experiment; 

^ velue of 99 mean- distribute devices as evenly as possible 

int ir.3VO_lis»ie ; /* maximum number of devices that may be moved */ 

int gen_mcu«; /* 0. 1, 2, or 3; selects method of generating ir.it iai device 

values V 

int num_trials; /* number, of trials (time segments) tc simulate */ 

double Vdev[:«AXpath5j tJOXdev] ; /• Vdev[i][jl is the value of the jth device on 
the Lth path »/ 

int Mdsv^MAXpachs] ; /' Kdev[i] is the number of devices assigned tc ith path V 

double Tl ; /* load is considered balanced when (hi - lc) /hi <- Tl, where hi 

and lo are the largest and smallest path values V 
double T2; /* a device is moved from hi oath to io path if new (hi -lei < Tl * 
eid(hi-io) ■/ 

/* hi = largest of ail path values */ 

double AgglnitHigh; / * aggregate initial hi of ail experiments V 

double Agg rir. a iKigh; /* aggregate final hi of all experiments */ 

int Nrtovos ; / * total number of devices moved in ail experiments V 

int num_reached_limic ? /* number of experiments that reached cnove_limit V 



long seed; /* for randcr.-number generator rand_unif ■! } */ 



, * £>;c C? EXTij\NAL variables •/ 



BEGINNING 0? nam(i 



tr.air.linc argc, c.:ar 'argvC! ) 
i 

int i; 



nvr\_pschs = atoi iargv-; I] ) : 
nu.TL.-av = arci (argv-12 ] ; ; 

dev_^par_tatli = atci (argv[3 j } ; /* 99 means distribute devices evenly * / 
Tl = siccc (argv(4! ) ; /* enter a percentage V 
V2 - atc£ CargviS] J ; / * enter as a percentage */ 
ucve_limit = atoi (argvt 6) ) ; 

gen_moce = atci ( argv[7 ] ) ; /* 0, 1, 2, or 3 V 
nunutriais = atci £arsv[9 3 ) ; 
seed = '-atoi iargv(9 I ) ; 



if (num_pachs > KAXpathS) ( 

prinLfCTCO KANY PATHS \ a- ); 
exit(0; ; 

V 

if (dev_per_path 1= 9?) { 

i- (nuir w dev < nu7\_pa.hs * dcv_p<3r_?ath} { 
princf ( "NOT ENO'JGH DEVICSSNa* ) ; 
ax«t(0J ; 

} 



print? ( - \naAiAXiCING %d DEVICES OX %d PATHS \n" f num_dev f mim_paths) ; 
prince ( •Threshold fsr judging balance = % . llf ,T1) ; 

print f ( "Threshold for judging whether a move is worthwhile = % . 11 £%%\n" , T2) ; 
print* ( 1 Maximu» nunber of devices that tar. be noved « %d\r.'- ,rtove_ii_Tiit j ; 
vl /* 100; /* convert from percentage to fraction 
72 /= 100; ;» convert from percentage to fraction V 
if (dev_per_path == S3) ( 

princf { "Devices distributed as evanly as possible . \n" ) ; 

els a ( 

print f { "At least %d devices per TaathAn" , dev_ser _cath) ; 

} 

ir (gen^nods 0 ) { 



print f « "Davica values are CONSTANT . \r.° 



* ; 



j 

else if (ger.^raode == I) f 

printf ( -Devica values are VNIFOFJULY distributed. \n <) ; 

} 

*ls* if ;gen_rtcde ==2) [ 

printff "Device values are EXPONENT I ALL Y distributed . \n" ) ; 

) 

*i32 if [g encode 3 ) { 

print f ( "Device values are based on PCWEP.-LA'.tf ftneticr. of rank. \n\' ; 

j 

else ( 

prir.tS ! " INVALID gan^mcde\n " ) ; 



) 

princf ("Mcjiber of trials = Hd; seed = %ld\n" , nun.zriala , -aeed} ; 
Amoves =0; 

r_ur*_r 2a cned_l imi c = C; 
Agg7inalKigh = 3; 
Asgl.nicHigh = C; 

:cr (i =0; i < nu.T._crials; i--) • 

/" generate device values and assigrunar.r of davices co paths V 
gsn_ values () ; 

/* balar.ee lead */ 
balance ■ ) ; 

/* crinz results V 

nrincf ( "Mean final high/rr.ean inicial nigh = 
a.21£%%\n" f ICO'AgcP^nalHigh/AgglnizHigh) ; 

prir.tf ("Mean number of devices moved - %. 31S\n" , (double )Kn»v33/nuin_tria la ! ; 
princf ( "KumbQr oz expariraants that raachad mova lirait = 
%d\n" . nuitur eachec_ 1 iird t ) ; 

) 



iiH^Tt**»*»iit«it»titi 3EGINNI2fG Or cer«_VS lues ( ) «»v*'»*tr , #***>* , »T*t****»T*^* 

"/ 

This function sets up the condicior.s for each experiment; that ia , it 
generates device values for a civ&r. ciraa sagnwnt- */ 

gen.values ( ) 

( 

inc extra, check; /* variables used in assignment of devices cc paths */ 
inc separator [XAXca fchaj ; /* array used in assignment of devices to paths */ 
ir.c nunusepararors; /* number of separator C] elements chat have teen ccrnpuced 
inc s?rted_5ep [MAXnaths } ; / T sorted version cf separator]] x / 
int sraai Les c , last; /** variables used in sorting saparacoi( ) " ' 
inc i,j,*; 

f* following 4 variables are used 10 generate random pemutaticn of slcev/ rank 

when g e r._:ro da = 3 */ 

inc pernuce_scurze[ 1GQC] ; 

inc permute^si za; / v numhar of unused elanents in parir.uta_source [ V 
inc index; 
double ran!<; 

double rand__e:cp ■ ) ; / * used for gan_xode = 2 x / 
double rand_uni £ ( ) ; 

fcr (i = 0; i < MAXpaths ; i++) { 
fur (j » 0; j < JGAXdev; C 
VdsvCi: [j] = 0; 

} 



for (i = 0; i < XAXpaths; i — i { 
Mdev[i! = C; 

} 

assign devices zc paths V 
iZ [devjer^path == 99) J 

/• devices a:s to be distributed as uniformly as 

possible ever the pacha V 
dev_per_path = nur^dsv/nun ^pdths; 
extra - r.or^cav - p.ua_paziis * dev_pQr_path; 
for (i m 0; i < extra; L+-) { 
fCdsvlil - dav_per_path + 1; 

} 

for (i = extra; i < nusupaths ; { 
N'd-v * i ] ^ dev_?e r_p a ch ; 

} 

) 

eis* { 

for Ci = 0; i < nunupatlis: i«i { 
NdsvCi] = dev_&c»r_pcith; 

) 

sxtra = nuaudev - nuflupaths * dev_per_path; 

/* randomly assign tha extra davic9S to path3 V 

ax era nusupaths - 1; 

for (nun_saparators = C; nunusecarators < num_path3 - 1; t-*nux_9eparators J ( 
separator [riunus operators] = r5Jic w u--if ( • * extra; 
for (i =0; i < r.uir._ sapara-ors ; i--) ( 

if . (separator [i ] « separator {nurr_separators ; ) { 
- -nunLSGparators ; 
break; 

} 

> 

} 

/* sore separator U */ 



Last = -1; 

-or (i = 0; L < n*-au?aths - 1; i- 
srnalLea - » extra; 

for (j = 0; j < r.urupaths - 1? j*-) { 

if (separator [j ; < smallest && separator [j! > last} [ 
smallest = separator !j ] ; 

} 

sorted^sapri; = smallest; 
last = srallest; 

} 

uae the separators to distribute the extra devices tc the par. ha */ 
Mggv'O ] t-= aorted_sap[0] ; 
Scr :i = 1; i < nux^paths - L; i^+) { 

Ndev ( i ] *s gcrtsd^aepf i] - 1 - scrtecUsep [ i-i; ; 

} 

Mdesv(num_path3 - 1] «•= e>:tra - : - sorted.ssp [nun_paths - 2]; 



/ x chacfc assigrjnent of devices to paths */ 
checfc - 0; 

for ( i = f J ; t < nun_paths; ( 
if :tf-4ev(i! > aiAXdev} { 



f 



♦ 



crinrf fTOO MAtfY mVICtS CM PATHXn* ) ; 

<=:<ic (0: ; 

} 

check Mdav [ i j ; 

> 

if (chack 1= aurt^cev) { 

prir.Lf ( * EPS.C?. IN ASSIGNMENT OF DEVICES TO PATHS Nr." I ; 

\ ) 

f~ compute -he device values •/ 
if (gar._r.cde 1) -; 

for (i =• C; i < num_jpath6; i* + ) { 

for (: = 0; j < Ndevfi]; j**) { 
Vdevt i) [ j ] = rancLunir ( } ; 

) 

} 

> 

else i 5 - (gen_mode «» 2) { 

for (i a Q; i <: num_paihs; i*-) f 
for (j =0; j < Xdev[iJ; j-r-) { 
Vdevtilfj: = ran4_exp(); 

} 

) 

} 

else if (gen_mode --3) { 

/* Use f omul a. tor s.'cew as function of rank given by ^ererson and Grossman in 
'Power Laws in Large Shop DASD I/O Activity-, CMG Proceedings for 1395. p.p. 
322-833. Figure 1; assign ranlcs to devices by generating a random 
permutation. •/ 

fcr (i = 0: I < numjev; irri { 
parmuro_sourc9 [i] = i + 1; 

} 

?errtur.e_sira * nunud*v; 
for (i = 0; i < nunvjaths; i*+) { 
for (j = 0/ j < NdevCi]; j**! ( 

index = rand_uni£() * pemuta^size; 
r»r.:< = permu te^sou re e C index } ; 
Vcev[i] [j] = pow{rarJ< * 1.94, -1.25); 

for (k = index/ k < perxute_sise; k++) { 

pemute_scurce [ k] = permute_source[k* 1* ; 

) 

elsa if (gan_mcde == 0) { 

This option v/as implemented fcr ces ring- purposes . In particular, cnis 
program run with a sufficiently large number of trial3 fcr num^paths » 2, 
nu.T_dev even. dev_per_par,h = 0, and gen_mode = 0 yields results that agree 
with the following expected solution: 

;mean final hi)/ (mean initial hi) = 2?DT r 1)/{2M t i) 
mean number of moves - 0.25tf(N + 2)/(N t 1), 
where M stands for nurv_dav. 

'/ 

for (i - 0; 1 < num_paths; i-f} { 



t 



for (: ^ 0; j * 2Cdev£ij ; j+r) 
Vdev(i) [j] = 0.1 ;i 

} 



/ * »#*********~*-.*.'*»*-«~*^ 3ZGIMTJING OJ balance ( ) */ 

balanced 
( 

double Vpath/MAXpazhs] i /* '/path!!] ia the toral value cf ths devices on ihe Lzh 
path */ 

cauJblQ hi? /* largest of all Vpath[i) */ 
double 1c; /* smallest of all V£ath[il */ 

int nunuaoved; /* nurcber of devices r.oved in current experiment V 
int hijath; /* cha path wiih V£ath£i] = hi V 
inc lo^pach; /* ch* pach wish V£ath[ii ■ lo V 

double trgc; /* crgc 13 the "optimum" device value to be noved from hi pach zo 
la pazh; 

algorithm uses" trgt = (hi-lo)/2 */ 
int trgt_dev; /* the next device to be movsd has valua Vdev(hi_path] (crgz_dev) 
*/ 

double x, delta; /« variables used in finding trgt_dev */ 
ir.t i,j; 

nor (i = 0; i < nuir._oaths ; irt] { 
VpathCi] = 3; 

for (i = 0; j < ECdev[i} ; { 
Vta:h(i; -= vdev[i] [j] * 



ic - 9S999S999S999999/ 

for [i a 0; i < nucupachs ; !-»--) { 
lo = m«n(lo, Vpathii]); 



hi = 0; 

for (i =0; i < nuKi_paths ; if- } ( 
hi » ma:c ( hi . Vpaih [ i 3 } ; 



Ac-jInicHigh r= hi; 
nu.-n_rr.oved a G ; 

whila (hi - la > TL * hi && raia^jnaved < mo vs_ limit) { 
erg- - {hi - lo) J 2; 

/* id*r.*:ify tho hi and lo paths */ 
for (i = C; i < nuKL_patlis; i*+i ■ 
if (Vpachfi] == hi! C 



hi _pa th = i ; 
break; 

i 

if (i >= r.urupalhsl { ■ 

princf i»g3«oa IN FINDING HXT?.2>:E VX2K\r. n > ; 
exitdi ; 

} 

for (i = 0; i < minv^pazhs; i + + ) ( 
if [Vpathji; loi { 
Io_path = i; 
break; 

j 

if (i >= r.-^_parhs) { 

print £ ; :, E?^.OR IM FINDING EXTREME ?A?H\n M ) ; 
cxit(l) ; 

} 

/ * identify -he next devices to be roved, if there is one h / 
daica = 9929993999995999; 
trgt_dev « Kcev Ifci^path] ; 
for Ci = 0; i < rcdev[hi_path] : ir+J { 
x = Vdav;hi_path] [i] ; 

if ifabsCx - trgc) < 72 * zrgz &£ fatsfx - erg-) < deLca) { 
trgt_dev = i; 
del^a - fab 3 (x - crgc) ; 

} 

} 

if (crgt.dsv == \ r dav [hijpath] ) (, 

/* hi pach contains r.c devic-s -hat car. profitably be moved 7 terminate 
balancing */ 

break; 

) 

/* we have found the device we wi3h to move */ 
/* add this dovics Co lo path V 

VievCio^path] [Ndevf lo_pat:h] ] = Vdev[hi_pauh] [trgt_dev] ; 
v?azhClc_pszh] r- 7dQv[hi_pachJ C trgt_d*v] ; 
*^Ndev ; lo_path| ; 

/* remove device from hi path */ 
V?a-h[hi_pa-hj Vdev[hi_path] icrgc_dev] ; 
--Ndev.;hi_path; ; 

for (i = trgc_dev; i < Ndev [hi_path] ; { 
Vdev[hi_path] [i] * Vdev[hi _path] [L*L]i 

} 

Vd«v[hrjath] [Nd*vfhi_path] ] = 0; 

/* conjputa new values of hi ar.d lo */ 
lo 99593S99S9999S99; 
for (i = C ; i < nun_paths; if * ) { 
lo » .-air.Jio, Vpachli]); 

} 



hi = Q ; 

for (i = 0; i < nuitupatha ; i 



hi = cr.sx(ci, Vpathf il ) ; 



) 

if | njir.^ov«d >- rr.ove_lin.it | ( 
+ * rv-.r._r cached.. Liriit ; 

) 

Mmove3 ► nura_ moved; 
AggFir-dlKigh hi? 



/* t ******** * * * * * * jr ********* * * £XD Or balanced! + */ 



/ * y*-^***s«; vi**^*),***^ 3EGI?JNXriG C3* rancL_exp {) ******************** * / 

/* This cur.cticr. is taken iron page 237 of "Numerical Recipes in C" 
i second edition) by Press, Tsukciskry, Veirerling, and Flanr.ery; it 
returns an exponent ial deviara with mean 1.0 calculated from a uniform 
dsviaca garva raced by rand_unif ( i . V 

double rand_ex? ( ) 
[ 

dcubl s- r dncL-un i f £ ) ; 
doubia durfl ; 

do { 

dun = rand_uni£{); 
) while (dum == Q.0); 

return -log (dum); 
} 

/* ********** »«*•+******»*«** CF rar.d_exp ( ) *************************** */ 



/ * ******** *** 3EGIHEING C? rand_urii£() A*********************** */ 

/* This function is taken fron page 230 of •NUrr.srical Recipes in C 
(second edition; by Press, Teukolsky, Vetterling, and Flannery; it 
returns a uniform deviate in (0,1). >/ 

double rand_uni z ( ) 

mt j; 
long k; 

static leng iy = 0L; 
static long iv[NTA3] ; 
double ceirp; 

it iseec <= CL ]| ! iyj ( /* initialize */ 

if (-seed < 11} { / * Se sure to prevent seed = 0 */ 

seed = IL; 

else ( 



ssad = -s*ed; 

fcr (j = f - j >= g ; j — ) ( / * Load chs sfcui'le table af-er a warmups 

k » seed/IQ; 

seed = IA*(:seed - k*IQ} - ia*!c; 
if (se*d < ol; ( 
seed -r= IM; 

i 

if £j < NT A3) { 
ivlj] = seed; 

} 

iy = iv[0] ; 

) 

k; = seed/ IQ; / * Start: here wher. nod initializing * / 

seed = IA* (seed - k*IQ) - IH*k; 
Lf (seed < 01) { 
soed *=■ IX; 

} 

j = iy/NDIV; 
iy = iv[ j ] ; 
ivTjj =s seed; 

if {{temp. - Atf*iy) > 3N>:X) t 

return RMMX; /* Because users don't expect andpoint valuas * 

} 

ei3e i 

return temp; 

j 
* 

^ * ^jrj qj rand_ur.i.f ( I ************************** */ 



